Method and apparatus for processing an image

ABSTRACT

The present invention provides a method of processing an image, which includes the steps of identifying one candidate for human face region within the image, selecting a mouth neighborhood within the candidate for human face region, processing the mouth neighborhood, and classifying the candidate for human face region based on results of the processing step. According to the method of the present invention, human faces are detected just based on pixels included in the mouth neighborhood, but not all the pixels of the entire face.

FIELD OF THE INVENTION

The present invention relates to image processing, and particularly tothe method and apparatus for processing an image in which human faceswill be detected.

BACKGROUND OF THE INVENTION

A number of techniques are known for detecting areas of interest in animage, such as a face or other identified object of interest. Facedetection is an area of particular interest, as face recognition hasimportance not only for image processing, but also for identificationand security purposes, and for human-computer interface purposes. Ahuman-computer interface not only identifies the location of a face, ifa face is present, it may also identify the particular face, and mayunderstand facial expressions and gestures.

Many studies on automatic face detection have been reported recently.References for example include “Face Detection and Rotations Estimationusing Color Information,” the 5th IEEE International Workshop on Robotand Human Communication, 1996, pp 341-346, and “Face Detection fromColor Images Using a Fuzzy Pattern Matching Method,” IEEE Transaction onPattern Analysis and Machine Intelligence, vol. 21, no. 6, June 1999.

All the conventional methods of detecting human faces have their ownadvantages as well as shortcomings depending upon different algorithmsused for processing images. Some methods are accurate but are complexand time-consuming.

SUMMARY OF THE INVENTION

The objective of the present invention is to provide a method andapparatus for processing an image in which human faces will be detectedbased on the pixels located in the mouth neighborhood, and specificallybased on the edge information calculated in relation to the mouthneighborhood.

For the attainment of the above objective, the present inventionprovides a method of processing an image, characterized by comprisingsteps of:

-   -   identifying one candidate for human face region within said        image;    -   selecting a mouth neighborhood within said candidate for human        face region;    -   processing said mouth neighborhood;

classifying said candidate for human face region based on results ofsaid processing step.

The present invention also provides an apparatus for processing animage, characterized by comprising:

a candidate identifier, for identifying one candidate for human faceregion within said image;

a mouth neighborhood selector, for selecting a mouth neighborhood withinsaid candidate for human face region that has been identified by saidcandidate identifier;

a mouth neighborhood processor, for processing said mouth neighborhoodthat has been selected by said mouth neighborhood selector;

a classifier, for classifying said candidate for human face region thathas been identified by said candidate identifier based on outputs ofsaid mouth neighborhood processor.

According to the method and apparatus of the present invention, humanfaces are detected just based on pixels included in the mouthneighborhood, but not all the pixels of the entire face. And the use ofedge information in relation to the mouth neighborhood increases theaccuracy of detecting human faces.

Additionally, the method of the present invention can be easily combinedwith various conventional methods of determining candidates for humanface regions so as to fit in different situations.

Other features and advantages of the present invention should beapparent from the following description of the preferred embodiments,taken in conjunction with the accompanying drawings, which illustrate,by way of example, the principles of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flow chart of the first embodiment of the method ofprocessing an image according to the present invention;

FIG. 2 is a flow chart of the second embodiment of the method ofprocessing an image according to the present invention;

FIG. 3A is a block diagram schematically shows the structure of theapparatus for processing an image according to the present invention;

FIG. 3B schematically shows the inner structure of the mouthneighborhood processor shown in FIG. 3A;

FIG. 3C schematically shows the inner structure of the edge informationcalculator shown in FIG. 3B;

FIG. 3D schematically shows another inner structure of the edgeinformation calculator shown in FIG. 3B;

FIG. 4 schematically shows the relationship among mouth neighborhood,mouth area and eye areas within a human face;

FIG. 5 shows two examples for explaining the methods shown in FIGS. 1and 2; and

FIG. 6 schematically shows an image processing system in which eachmethod shown in FIGS. 1 and 2 can be implemented.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

The present invention will be described in detail. In the followingdescription, as to how to identify one candidate for human face regionwithin an image, reference can be made to Chinese Patent Application No.00127067.2, which was filed by the same applicant on Sep. 15, 2000, andmade public on Apr. 10, 2002, and Chinese Patent Application No.01132807.X, which was filed by the same applicant on Sep. 6, 2001. Theseapplications are incorporated here for reference. However, the method ofidentifying candidates for human face regions disclosed in ChinesePatent Applications No. 00127067.2 and No. 01132807.X constitute norestriction to the present invention. Any conventional method ofidentifying candidates for human face regions within an image may beutilized in the present invention.

FIG. 1 is a flow chart of the first embodiment of the method ofprocessing an image according to the present invention.

The process begins at step 101. At step 102, an image to be processed isinputted. At step 103, one candidate for human face region is identifiedwithin the image inputted at step 102. The size of the candidate forhuman face region identified at step 103 is denoted as S1. Here, thesize of an image is defined as the number of the pixels composing theimage.

In steps 102 and 103, any conventional methods of identifying onecandidate for human face region within an image can be adopted, andconstitute no restriction to the present invention.

At step 104, a mouth neighborhood is selected within the candidate forhuman face region identified at step 103. As to how to select the mouthneighborhood within one human face, detailed description will be givenlater with reference to FIG. 4.

Then, the mouth neighborhood is processed.

Two different methods of processing the mouth neighborhood are shown atsteps 105 to 111 enclosed in the broken line in FIG. 1, and at steps 205to 207 enclosed in the broken line in FIG. 2. The common ground for thetwo different methods shown in FIGS. 1 and 2 is that an edge map isfirst formed for the candidate for human face region or at least for themouth neighborhood; edge information is calculated for the mouthneighborhood based on the edge map; and finally the edge information isused for classifying the candidate for human face region as onecandidate with high possibility of being a real human face, onecandidate with high possibility of being a false human face, a realhuman face or a false human face.

Of course, other methods of processing can be applied to the mouthneighborhood as long as the results of the processing of the mouthneighborhood will be sufficient for classifying the candidate for humanface region as one candidate with high possibility of being a real humanface, one candidate with high possibility of being a false human face, areal human face or a false human face. And other edge information, inaddition to the size of special areas (e.g., bright areas formed at step106 in FIG. 1) and the average edge intensity within an area (e.g., thetwo intensities calculated at step 206 in FIG. 2), can be used in themethod of processing the mouth neighborhood as long as the edgeinformation will be sufficient for classifying the candidate for humanface regions as one candidate with high possibility of being a realhuman face, one candidate with high possibility of being a false humanface, a real human face or a false human face.

Different methods of processing the mouth neighborhood and differentedge information do not constitute any restriction to the presentinvention.

Specifically speaking, in FIG. 1, at step 105, an edge map is formedwhich at least corresponds to the mouth neighborhood.

It should be noted that if the entire candidate for human face region isconverted into an edge map, the order of steps 104 and 105 will not becritical.

That is, an edge map can be formed for the entire candidate for humanface region and then a mouth neighborhood is selected within the edgemap.

If the candidate for human face region is represented as a gray leveldiagram, the formed edge map will show a plurality of bright edgesagainst a black background. Each bright edge in the edge map representsthat the gray level of the pixels at the corresponding place in the graylevel diagram of the candidate for human face region changessignificantly. In order to change a gray level diagram into an edge map,conventional “Sobel” operator can be utilized. At the end of thespecification, four examples are given in order to explain the methodsshown in FIGS. 1 and 2.

At step 106, from the edge map formed at step 105, a plurality of pixelswhose characteristic values are greater than a first threshold areselected. These selected pixels, at their own positions, compose aseries of bright areas within the edge map.

A unique method of selecting such pixels, referred to as “binarization,”will be illustrated in the four examples described at the end of thespecification.

Among the series of bright areas, the size of the biggest bright area isdenoted as S2, and the size of the second biggest bright area is denotedas S3. As usual, the size of an area is the number of the pixels locatedwithin the area. As to how to identify the biggest and the secondbiggest areas among the series of bright areas, a lot of methods can beutilized. For instance, an algorithm called “Labeling” is applicablehere, which is used in the four examples described at the end of thespecification.

Then, at step 107, it is decided whether the ratio of S2 to S1 (i.e.,S2/S1) is greater than or equal to a second threshold. The secondthreshold ranges from 0 to 0.1, and preferably takes the value of 0.003.The purpose of step 107 is to decide whether there is a prominent brightarea.

If the result of step 107 is Yes, the process goes to step 108; else tostep 112.

At step 112, the candidate for human face region is classified as afalse human face, or one candidate with high possibility of being afalse human face.

At step 108, it is decided whether the ratio of S3 to S2 (i.e., S3/S2)is less than a third threshold. The third threshold ranges from 0.2 to0.8 and preferably takes the value of 0.5. The purpose of step 108 is todecide whether there is a most prominent bright area.

If the result of step 108 is Yes, the process goes to step 109; else tostep 110.

If the process goes to step 109, it means that there is a bright areawhich is the most prominent among the series of bright areas. If so,this bright area must be the biggest bright area.

At step 109, it is decided whether the center of the biggest bright areais within the mouth area. The mouse area is a predefined area within themouth neighborhood. As to the definition of mouth area, detaileddescription will be given later with reference to FIG. 4.

If the result of step 109 is No, the process goes to step 112; else tostep 113.

At step 113, the candidate for human face region is classified as a realhuman face, or one candidate with high possibility of being a real humanface.

At step 110, the mid-point between the centers of the first two biggestbright areas is identified. Then, the process goes to step 111.

At step 111, it is decided whether the mid-pointed identified at step110 is within the mouth area.

If the result of step 111 is Yes, the process goes to step 113; else tostep 112.

After steps 112 or 113, the process goes to step 114, where the processof detecting a human face within an image is ended.

Once human faces have been detected within an image, further processingcan be conducted on the image, or the detected human faces.

FIG. 2 is a flow chart of the second embodiment of the method ofprocessing an image according to the present invention.

Steps 201-205 and steps 212-214 are similar to steps 101-105 and steps112-114 in FIG. 1 and detailed description of them are omitted.

As described above with reference to FIG. 1, the edge informationutilized in FIG. 2 is the average edge intensities of mouth neighborhoodand mouth area.

Specifically speaking, at step 206, the average edge intensities withinthe mouth area and the mouth neighborhood are respectively calculated,and denoted as I1 and I2. Then, at step 207, it is decided whether thedifference between I1 and I2 is greater than a fourth threshold. Thefourth threshold ranges from 0 to 50, and preferably takes the value of5.

If the result of step 207 is No, the process goes to step 212; else tostep 213.

FIG. 3A is a block diagram schematically shows the structure of theapparatus for processing an image according to the present invention.

As shown in FIG. 3A, reference numeral 301 represents candidateidentifier; 302, classifier; 303, mouth neighborhood selector; 304,mouth neighborhood processor.

Candidate identifier 301 is used to identify one candidate for humanface region with the image to be processed. Any conventional algorithmsfor identifying one candidate for human face region within an image canbe adopted in candidate identifier 301, and constitute no restriction tothe present invention.

Mouth neighborhood selector 303 is used to select a mouth neighborhoodwithin the candidate for human face region that has been identified bycandidate identifier 301. As how to select a mouth neighborhood within ahuman face, detailed description will be given later with reference toFIG. 4.

Mouth neighborhood processor 304, is used to process the mouthneighborhood that has been selected by mouth neighborhood selector 303,and outputs processing results, which are usually some sorts ofcharacteristic values of the mouth neighborhood selected by mouthneighborhood selector 303.

Classifier 302 is used to classify the candidate for human face regionthat has been identified by candidate identifier 301 as one candidatewith high possibility of being a real human face, one candidate withhigh possibility of being a false human face, a real human face or afalse human face, based on the outputs of mouth neighborhood processor304.

Usually, the outputs of mouth neighborhood processor 304 are some sortsof characteristic values of the mouth neighborhood selected by mouthneighborhood selector 303, if these characteristic values are sufficientfor classifier 302 to classify the candidate for human face region asone candidate with high possibility of being a real human face, onecandidate with high possibility of being a false human face, a realhuman face or a false human face.

Although it is shown in FIG. 3A as well as FIGS. 3B to 3D that thecandidate for human face region that has been identified by candidateidentifier 301 is inputted to classifier 302, it is not necessary to doso in practice. The important is that classifier 302 will know whichcandidate for human face region is to be classified when it receives theoutputs (e.g., characteristic values of one candidate for human faceregion) from mouth neighborhood processor 304.

The classification result of classifier 302 can be used for furtherprocessing of the image.

It should be noted that any methods of processing can be applied bymouth neighborhood processor 304 to the mouth neighborhood as long asthe results of the processing of the mouth neighborhood will besufficient for classifier 302 to classify the candidate for human faceregion as one candidate with high possibility of being a real humanface, one candidate with high possibility of being a false human face, areal human face or a false human face.

FIG. 3B schematically shows the inner structure of mouth neighborhoodprocessor 304 shown in FIG. 3A.

If edge information is utilized in the processing conducted by mouthneighborhood processor 304, mouth neighborhood processor 304 willcomprise at least two components, i.e., converter 305 and edgeinformation calculator 306, as shown in FIG. 3B.

Converter 305 receives mouth neighborhood or the entire candidate forhuman face region outputted by mouth neighborhood selector 303, andconverts at least the mouth neighborhood into an edge map. Edge map hasbeen described above.

Edge information calculator 306 is used to calculate edge informationfor the mouth neighborhood based on the edge map formed by converter305, and outputs edge information to classifier 302.

Depending upon different types of edge information, edge informationcalculator 306 has a lot of embodiments, two of which are shown in FIGS.3C and 3D.

FIG. 3C schematically shows one of the inner structures of the edgeinformation calculator shown in FIG. 3B.

As shown in FIG. 3C, edge information calculator 306 comprises brightarea identifier 307 and size calculator 308. Bright area identifier 307is used to identify within the edge map formed by converter 305 a seriesof bright areas each of which is composed of pixels whose characteristicvalues are greater than a first threshold.

A unique method of selecting such pixels, referred to as “binarization,”will be illustrated in the four examples described at the end of thespecification.

Size calculator 308, is used to calculated the size of the candidate forhuman face S1 and the size of the biggest bright area S2. If S2/S1 isless than a second threshold, classifier 302 will classify the candidatefor human face region as a false human face, or one candidate with highpossibility of being a false human face.

Size calculator 308 may also calculates the size of the second biggestbright area S3, among the series of bright areas identified by brightarea identifier 307. Then, if S3/S2 is less than a third threshold andthe center of the biggest bright area is not in a mouth area, classifier302 will classify the candidate for human face region as a false humanface, or one candidate with high possibility of being a false humanface. The mouth area is a predefined portion of the mouth neighborhood,and will be described in detail later with reference to FIG. 4.

If S3/S2 is less than the third threshold and the center of the biggestbright area is in the mouth area, classifier 302 will classify thecandidate for human face region as a real human face, or one candidatewith high possibility of being a real human face.

If S3/S2 is not less than the third threshold and the mid-point betweenthe centers of the first two biggest bright areas is not in the moutharea, classifier 302 will classify the candidate for human face regionas a false human face, or one candidate with high possibility of being afalse human face.

If S3/S2 is not less than the third threshold and the mid-point betweenthe centers of the first two biggest bright areas is in said mouth area,classifier 302 will classify the candidate for human face region as areal human face, or one candidate with high possibility of being a realhuman face.

FIG. 3D schematically shows another inner structure of the edgeinformation calculator shown in FIG. 3B.

As shown in FIG. 3D, edge information calculator 306 comprises an edgeintensity calculator 309, which is used to calculate the average edgeintensity within the mouth neighborhood (I2) and the average edgeintensity within the mouth area (I1).

If the difference (I1−I2) between the average edge intensity within themouth area and the average edge intensity within the mouth neighborhoodis not greater than a fourth threshold, classifier 302 will classify thecandidate for human face region as a false human face, or one candidatewith high possibility of being a false human face.

If the difference (I1−I2) between the average edge intensity within themouth area and the average edge intensity within the mouth neighborhoodis greater than the fourth threshold, classifier 302 will classify thecandidate for human face region as a real human face, or one candidatewith high possibility of being a real human face.

FIG. 4 schematically shows the relationship among mouth neighborhood,mouth area and eye areas (right eye area and left eye area) within ahuman face, or a candidate for human face region.

Reference 401 represents a human face; 402, mouth neighborhood; 403,mouth area; and 404, eye areas.

The width of human face 401 is W, and the height of human face 401 is H.

The width of mouth neighborhood 402 is W2, and the height of mouthneighborhood 402 is H1.

The width of mouth area 403 is W1, and the height of mouth area 403 isat most H1.

The width of eye areas 404 is W4, and the height of eye areas 404 is H2.

The space between eye areas 404 and the vertical border of human face401 is W3, and the space between eye areas 404 and the top border ofhuman face 401 is H3.

The above width and height satisfy the following equations:H1=H*r1, 0.1<r1<0.7W1=W*r2, 0.1<r2<0.7W2=W*r3, 0.2<r3<1W3=W*r4, 0<r4<0.2W4=W*r5, 0.1<r5<0.4H2=H*r6, 0.1<r6<0.4H3=H*r7, 0.2<r7<0.4

Preferably, the following is satisfied:

r1=0.3,

r2=0.5,

r3=0.7,

r4=0.125,

r5=0.25,

r6=0.25,

r7=0.3.

The selection of mouth neighborhood and mouth area conducted in FIGS. 1to 3 may follow the above equations.

FIG. 5 shows two examples for explaining the methods shown in FIGS. 1and 2.

EXAMPLE 1

As shown in FIG. 5, one candidate for human face region A1 is firstidentified. With respect to the gray level diagram of candidate forhuman face region A1, “Sobel” operator is applied, and an edge map A2 isobtained. This process corresponds to step 105 in FIG. 1, or step 205 inFIG. 2.

Next, a mouth neighborhood A7 is selected within edge map A2. Thisprocess corresponds to step 104 in FIG. 1 or step 204 in FIG. 2. Asdescribed above with reference to FIG. 1, the order of steps 104 and 105are not critical.

Then, a process called “binarization” is executed with respect to mouthneighborhood A7. This process corresponds to step 106 if FIG. 1.

In this process, a predetermined threshold is used. This threshold isreferred to as the first threshold in step 106 of FIG. 1. Here, thefirst threshold may take a fixed value. It is preferably that the firstthreshold ranges from 100 to 200. However, it would be more better toselect the value for the first threshold based on the average value ofthe edge intensities of two eye areas A4, A5 within the edge map A2.

Let the width of candidate for human face region A1 be 360, and theheight of candidate for human face region A1 be 450. The constants r1,r2, . . . , r7 take their preferred values. Thus,

H1=H*r1=450*0.3=135,

W1=W*r2=360*0.5=180,

W2=W*r3=360*0.7=252,

W3=W*r4=360*0.125=45,

W4=W*r5=360*0.25=90,

H2=H*r6=450*0.25=112.5, and

H3=H*r7=450*0.3=135.

Based on the above constants, mouth area A6, mouth neighborhood A7, eyeareas A4 and A5 are obtained as shown in FIG. 5.

Let the threshold for the “binarization” be:R9=(the average edge intensity of right eye area+the average edgeintensity of left eye area)/2 *r8.

Here, r8 is a proportional threshold, ranging from 0.4 to 13, andpreferably takes the value of 0.8.

Suppose the average edge intensity of right eye area A4 is 35, and thatof left eye area A5 is 31. Let r8=0.8. Then, r9=(35+31)/2*0.8=26.4.

After the “binarization” of mouth neighborhood A7, area A8 is obtained.

Then, a “labeling” process is executed with respect to area A8. The“labeling” process aims to calculate the number of the bright areaslocated in A8, the centers and sizes of each of the bright areas. Thisprocess also corresponds to step 106 in FIG. 1.

After the “labeling” process, it is calculated that there are threebright areas, i.e., A9, A10 and A11, as shown in FIG. 5. The centers ofA9, A10 and A11 are (165, 364), (216, 407) and (180, 397) respectively.The sizes of A9, A10 and A11 are 574, 490 and 728 respectively.

Then, it is decided whether there is a prominent bright area among thethree bright areas. The size of A1 is S1=360*450=162000. The size of thebiggest bright area which is A11 is S2=728. Then,S2/S1=728/162000=0.00449. Let the second threshold be 0.003. Then, S2/S1is greater than the second threshold. This process corresponds to step107 in FIG. 1.

Next, it is decided whether there is a bright area which is the mostprominent. Among A9, A10 and A11, the first two biggest bright areas areA9 and A11, and their sizes are 574 and 728 respectively. That is,S2=728 and S3=574. Then, S3/S2=574/728=0.7885. Let the third thresholdbe 0.5. Then, S3/S2 is greater than the third threshold. This processcorresponds to step 108 in FIG. 1.

Then, the mid-point between the centers of the first two biggest brightareas A9 and A11 are calculated. Since the centers of A9 and A11 are(165, 364) and (180, 397) respectively, the mid-point between A9 and A11is ((165+180)/2, (364+397)/2)=(172, 380). The position of the mid-pointis shown as A12 in FIG. 5. This process corresponds to step 110 in FIG.1.

Since the mid-point is located within mouth area A6, candidate for humanface region A1 is classified as a real human face, or one candidate withhigh possibility of being a real human face. This process corresponds tosteps 111 and 113 in FIG. 1.

EXAMPLE 2

Also as shown in FIG. 5, candidate for human face region B1 isidentified. With respect to the gray level diagram of candidate forhuman face region B1, “Sobel” operator is applied, and an edge map B2 isobtained.

A “binarization” process is executed with respect to mouth neighborhoodB4, and obtains B5.

A “labeling” process is executed with respect to B5, and it iscalculated that there is no bright area within B5.

Then, candidate for human face region B1 is classified as a false humanface, or one candidate with high possibility of being a false humanface. This process corresponds to step 112 in FIG. 1.

EXAMPLE 3

First, the candidate for human face region A1 is identified. Next, withrespect to the gray level diagram of candidate for human face region A1,“Sobel” operator is applied, and an edge map A2 is obtained. Theseprocesses correspond to steps 203 and 205 in FIG. 2.

Then, the average edge intensities of mouth area A6 and mouthneighborhood A7 are calculated as I1=28 and I2=20. This processcorresponds to step 206 in FIG. 2.

Then, it is decided whether the difference between I1 and I2 is greaterthan the fourth threshold, which preferable takes the value of 5. SinceI1−I2=28−20=8, which is greater than 5, candidate for human face regionA1 is classified as a real human face, or one candidate with highpossibility of being a real human face. These processes correspond tosteps 207 and 213 in FIG. 2.

EXAMPLE 4

First, one candidate for human face region B1 is identified. Second,with respect to the gray level diagram of candidate for human faceregion B1, “Sobel” operator is applied, and an edge map B2 is obtained.

Next, the average edge intensities of mouth area B6 and mouthneighborhood B7 are calculated as I1=0 and I2=0. This processcorresponds to step 206 in FIG. 2.

Then, it is decided whether the difference between I1 and I2 is greaterthan the fourth threshold, which preferable takes the value of 5. SinceI1−I2=0−0=0, which is less than 5, candidate for human face region B1 isclassified as a false human face, or one candidate with high possibilityof being a false human face. These processes correspond to steps 207 and213 in FIG. 2.

FIG. 6 schematically shows an image processing system in which eachmethod shown in FIGS. 1 and 2 can be implemented. The image processingsystem shown in FIG. 6 comprises a CPU (Central Processing Unit) 601, aRAM (Random Access Memory) 602, a ROM (Read only Memory) 603, a systembus 604, a HD (Hard Disk) controller 605, a keyboard controller 606, aserial port controller 607, a parallel port controller 608, a displaycontroller 609, a hard disk 610, a keyboard 611, a camera 612, a printer613 and a display 614. Among these components, connected to system bus604 are CPU 601, RAM 602, ROM 603, HD controller 605, keyboardcontroller 606, serial port controller 607, parallel port controller 608and display controller 609. Hard disk 610 is connected to HD controller605, and keyboard 611 to keyboard controller 606, camera 612, to serialport controller 607, printer 613 to parallel port controller 608, anddisplay 614 to display controller 609.

The functions of each component in FIG. 6 are well known in the art andthe architecture shown in FIG. 6 is conventional. Such an architecturenot only applies to personal computers, but also applies to hand helddevices such as Palm PCs, PDAs (personal data assistants), digitalcameras, etc. In different applications, some of the components shown inFIG. 6 may be omitted. For instance, if the whole system is a digitalcamera, parallel port controller 608 and printer 613 could be omitted,and the system can be implemented as a single chip microcomputer. Ifapplication software is stored in EPROM or other non-volatile memories,HD controller 605 and hard disk 610 could be omitted.

The whole system shown in FIG. 6 is controlled by computer readableinstructions, which are usually stored as software in hard disk 610 (oras stated above, in EPROM, or other non-volatile memory). The softwarecan also be downloaded from the network (not shown in the figure). Thesoftware, either saved in hard disk 610 or downloaded from the network,can be loaded into RAM 602, and executed by CPU 601 for implementing thefunctions defined by the software.

It involves no inventive work for persons skilled in the art to developone or more pieces of software based on one or more of the flowchartsshown in FIGS. 1 and 2. The software thus developed will carry out themethods of processing an image as shown in FIGS. 1 and 2.

In some sense, the image processing system shown in FIG. 6, if supportedby software developed based on flowcharts shown in FIGS. 1 and 2,achieves the same functions as the apparatus for processing image shownin FIGS. 3A to 3D.

While the foregoing has been with reference to specific embodiments ofthe invention, it will be appreciated by those skilled in the art thatthese are illustrations only and that changes in these embodiments canbe made without departing from the principles of the invention, thescope of which is defined by the appended claims.

1. A method of processing an image, characterized by comprising stepsof: identifying one candidate for human face region within the image;selecting a mouth neighborhood within the candidate for human faceregion; forming an edge map at least corresponding to the mouthneighborhood; calculating edge information for the mouth neighborhoodbased on the edge map; and classifying the candidate for human faceregion based on the edge information.
 2. The method of processing animage according to claim 1, characterized in that said step of selectingselects a mouth neighborhood which is r3 times the width of thecandidate for human face region, and r1 times the height of thecandidate for human face region, wherein r3 is a constant greater than0.2 and less than 1, and r1 is a constant greater than 0.1 and less than0.7.
 3. The method of processing an image according to claim 2,characterized in that r3 equals 0.7, and said r1 equals 0.3.
 4. Themethod of processing an image according to claim 1, characterized inthat said step of calculating edge information comprises steps of:selecting a portion of the edge map which corresponds to neighborhoodpixels whose characteristic values are greater than a first threshold,resulting in a series of bright areas composed of selected pixels;calculating the size S1 of the candidate for human face region and thesize S2 of the biggest bright area; and in that said step of classifyingcomprises a step of: classifying the candidate for human face region asa false human face, or one candidate with high possibility of being afalse human faces if S2/S1 is less than a second threshold.
 5. Themethod of processing an image according to claim 4, characterized inthat said step of calculating edge information further comprises a stepof: calculating the size S3 of the second biggest bright area; and inthat said step of classifying further comprises a step of: classifyingthe candidate for human face region as a false human face, or onecandidate with high possibility of being a false human face, if S3/S2 isless than a third threshold and the center of the biggest bright area isnot in a mouth area, the mouth area being a predefined portion of themouth neighborhood.
 6. The method of processing an image according toclaim 5, characterized in that said step of classifying furthercomprises a step of: classifying the candidate for human face region asa real human face, or one candidate with high possibility of being areal human faces if S3/S2 is less than the third threshold and thecenter of the biggest bright area is in the mouth area.
 7. The method ofprocessing an image according to claim 4, characterized in that saidstep of calculating edge information further comprises a step of:calculating the size S3 of the second biggest bright area; and in thatsaid step of classifying further comprises a step of: classifying thecandidate for human face region as a false human face, or one candidatewith high possibility of being a false human faces if S3/S2 is not lessthan a third threshold and the mid-point between the respective centersof the first and second biggest bright areas is not in a mouth area, themouth area being a predefined portion of the mouth neighborhood.
 8. Themethod of processing an image according to claim 7, characterized inthat said step of classifying further comprises a step of: classifyingthe candidate for human face region as a real human face, or onecandidate with high possibility of being a real human face if S3/S2 isnot less than the third threshold and the mid-point between therespective centers of the first and second biggest bright areas is inthe mouth area.
 9. The method of processing an image according to claim1, characterized in that said step of calculating edge informationcomprises a step of: calculating the average edge intensity within themouth neighborhood and the average edge intensity within a mouth area,the mouth area being a predefined portion of the mouth neighborhood; andin that said step of classifying comprises a step of: classifying thecandidate for human face region as a false human face, or one candidatewith high possibility of being a false human face if the differencebetween the average edge intensity within the mouth area and the averageedge intensity within the mouth neighborhood is not greater than apredefined threshold.
 10. The method of processing an image according toclaim 9, characterized in that said step of classifying furthercomprises a step of: classifying the candidate for human face region asa real human face, or one candidate with high possibility of being areal human face, if the difference between the average edge intensitywithin the mouth area and the average edge intensity within the mouthneighborhood is greater than the predefined threshold.
 11. The method ofprocessing an image according to claim 5, characterized in that themouth area is r2 times the width of the candidate for human face region,and at most r1 times the height of the candidate for human face region,wherein r2 is a constant greater than 0.1 and less than 0.7, and r1 is aconstant greater than 0.1 and less than 0.7.
 12. The method ofprocessing an image according to claim 11, characterized in that r2equals 0.5, and said r1 equals 0.3.
 13. An apparatus for processing animage, characterized by comprising: a candidate identifier, foridentifying one candidate for human face region within the image; amouth neighborhood selector, for selecting a mouth neighborhood withinthe candidate for human face region that has been identified by saidcandidate identifier; a converter, for converting a portion of the imageincluding at least the mouth neighborhood into an edge map; an edgeinformation calculator, for calculating edge information for the mouthneighborhood based on the edge map; and a classifier, for classifyingthe candidate for human face region that has been identified by saidcandidate identifier based on the edge information.
 14. The apparatusfor processing an image according to claim 13, characterized in thatsaid mouth neighborhood selector selects a mouth neighborhood which isr3 times the width of the candidate for human face region, and r1 timesthe height of the candidate for human face region, wherein r3 is aconstant greater than 0.2 and less than 1, and r1 is a constant greaterthan 0.1 and less than 0.7.
 15. The apparatus for processing an imageaccording to claim 14, characterized in that r3 equals 0.7, and said r1equals 0.3.
 16. The apparatus for processing an image according to claim13, characterized in that said edge information calculator comprises: abright area identifier, for identifying within the edge map a series ofbright areas each of which is composed of pixels whose characteristicvalues are greater than a first threshold; a size calculator, forcalculating the size S1 of the candidate for human face region and thesize S2 of the biggest bright area; and in that said classifierclassifies the candidate for human face region as a false human face, orone candidate with high possibility of being a false human face, ifS2/S1 is less than a second threshold.
 17. The apparatus for processingan image according to claim 16, characterized in that said sizecalculator also calculates the size S3 of the second biggest brightarea; and in that said classifier further classifies the candidate forhuman face region as a false human face, or one candidate with highpossibility of being a false human face, if S3/S2 is less than a thirdthreshold and the center of the biggest bright area is not in a moutharea the mouth area being a predefined portion of the mouthneighborhood.
 18. The apparatus for processing an image according toclaim 17, characterized in that said classifier further classifies thecandidate for human face region as a real human face, or one candidatewith high possibility of being a real human faces if S3/S2 is less thanthe third threshold and the center of the biggest bright area is in themouth area.
 19. The apparatus for processing an image according to claim16, characterized in that said size calculator also calculates the sizeS3 of the second biggest bright area; and in that said classifierfurther classifies the candidate for human face region as a false humanface, or one candidate with high possibility of being a false humanface, if S3/S2 is not less than a third threshold and the mid-pointbetween the respective centers of the first and second biggest brightareas is not in a mouth area, the mouth area being a predefined portionof the mouth neighborhood.
 20. The apparatus for processing an imageaccording to claim 19, characterized in that said classifier furtherclassifies the candidate for human face region as a real human face, orone candidate with high possibility of being a real human face, if S3/S2is not less than the third threshold and the mid-point between therespective centers of the first and second biggest bright areas is inthe mouth area.
 21. The apparatus for processing an image according toclaim 13, characterized in that said edge information calculatorcomprises: an edge intensity calculator, for calculating the averageedge intensity within the mouth neighborhood and the average edgeintensity within a mouth area, the mouth area being a predefined portionof the mouth neighborhood; and in that said classifier classifies thecandidate for human face region as a false human face, or one candidatewith high possibility of being a false human face, if the differencebetween the average edge intensity within the mouth area and the averageedge intensity within the mouth neighborhood is not greater than apredefined threshold.
 22. The apparatus for processing an imageaccording to claim 21, characterized in that said classifier furtherclassifies the candidate for human face region as a real human face, orone candidate with high possibility of being a real human face, if thedifference between the average edge intensity within the mouth area andthe average edge intensity within the mouth neighborhood is greater thanthe predefined threshold.
 23. The apparatus for processing an imageaccording to claim 17, characterized in that the mouth area is r2 timesthe width of the candidate for human face region, and at most r1 timesthe height of the candidate for human face region, wherein r2 is aconstant greater than 0.1 and less than 0.7, and r1 is a constantgreater than 0.1 and less than 0.7.
 24. The apparatus for processing animage according to claim 23, characterized in that said r2 equals 0.5,and said r1 equals 0.3.